Ranking from Pairwise Comparisons: The Role of the Pairwise Preference Matrix

Ranking a set of items given preferences among them is a fundamental problem that arises naturally in several real world situations such as ranking candidates in elections, movies, sports teams, documents in response to a query to name a few. In each of these situations, the preference data among the items to be ranked may be available in different forms: as pairwise comparisons, partial orderings or complete orderings. Ranking from pairwise comparison data is especially attractive in large scale settings (such as movie ranking) where the number of items to be ranked is large and it is impractical to obtain partial or complete rankings over the items. Moreover studies in the cognitive psychology literature confirm that it is easy for humans to give preferences among items by comparing them in pairs as opposed to giving complete orderings over a set of items. While several algorithms (both classical and recent) have been proposed for the problem of ranking from pairwise comparisons in multiple communities such as machine learning, operations research, linear algebra, statistics [23, 13, 12, 18], it is not well understood under what conditions these different algorithms perform well. In this thesis, we aim to fill this fundamental gap in understanding by studying the properties of different algorithms for ranking from pairwise comparisons and proposing new algorithms which have better statistical guarantees than previous algorithms. Towards this, we will consider a natural framework to study this problem wherein for every pair of items (i, j), there is a coin with bias pij that is flipped to decide the outcome whenever items i and j are compared. The matrix P containing all these numbers pij is called the pairwise preference matrix and has been used either implicitly or explicitly in much previous work in this area [2, 18, 24]. In this thesis, will elucidate the crucial role played by this matrix in determining the performance of various existing algorithms under different settings and in developing new algorithms with improved statistical guarantees.